Systems and methods for managing multimodal documents

ABSTRACT

A multimodal document management system and method includes a context manager for managing multimodal documents. A virtual media content management system is coupled to the context manager and a physical media content management system is coupled to the context manager. The context manager is operable to issue multimodal document management instructions to the virtual media content management system and issuing instructions to the physical media content management system. The context manager instructions may relate to shredding of multimodal documents by deleting virtual media and destroying physical media. The method for managing multimodal documents may include receiving a request for the shredding of an information and content data file containing a list of virtual media and physical media relating to a multimodal document. A determination is made if the request for shred is authorized and, where said request for shred is authorized, a record of the shred request is created. Elements in the element list are grouped based on elements in the list which are managed by the same content management system. A message is sent to each content management system requesting the shredding of each element for the grouping of elements for the content management system. The content management systems may verify and report the success of the shredding.

RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. 119(e) of U.S.provisional patent applications: Ser. No. 60/909,281 filed Mar. 30,2007, and entitled “A METHOD AND SYSTEM FOR ENABLING COLLABORATIVECAPTURE AND REPLAY OF DIGITAL MEDIA FILES USING PHYSICAL DOCUMENTS”; andSer. No. 60/909,273 filed Mar. 30, 2007, and entitled “SYSTEMS ANDMETHODS FOR MANAGING MULTIMODAL DOCUMENTS.” Both provisional patentapplications are incorporated herein by reference.

FIELD OF THE INVENTION

The invention relates generally to multimodal documents and moreparticularly to systems and methods for managing multimodal documentsincluding the shredding of multimodal documents through the deletion ofvirtual media and the destruction of physical media.

BACKGROUND OF THE INVENTION

Document retention and destruction is becoming increasingly important.Government regulations and litigation threats place pressure oncompanies to properly handle documents that have reached the end oftheir usefulness. Additionally, documents that are not properlydiscarded can contribute to corporate espionage and identity theft. Thehandling of documents is also complicated by the multimedia anddistributed nature of documents in our environment. Multimedia documentsare documents that contain or are associated with different types ofcontent or media. For example, a document may exist which could consistof word-processing content, video annotations, or other types of media.This media although associated with the document, is not necessarilyembedded in the document and can be found in multiple locationsthroughout the environment.

Various systems have been developed for deleting and destroyingdocuments and related content. U.S. Pat. No. 5,897,643 for SYSTEM FORMAINTAINING A RECORD OF DOCUMENTS INCLUDING DOCUMENT COPIES discloses asystem that maintains a listing of a document and its copies where theelectronic record of a copy is deleted when that copy is deleted. U.S.Pat. No. 6,839,707 for WEB-BASED SYSTEM AND METHOD FOR MANAGING LEGALINFORMATION discloses a web-based application that organizes the storageof patent applications and other legal documents into a hierarchicalfolder structure. It allows for the deletion of individual documentswithin a folder and also the deletion of the folders themselves. U.S.Pat. No. 6,148,312 for METHOD AND SYSTEM FOR ELECTRONIC ARTICLEMAINTENANCE AND DELETION discloses a system for deleting documents froman electronic document management system. Requests for deletion must beconfirmed by the creator of the document or a pre-set reviewer. Multipledelete sites are possible and are responsible for the confirmation andcorresponding deletion of the document and its metadata. Additionally,paper entitled “A Multimedia Document Filing System” by Xien Fan,Qianhong Liu, and Peter A. Ng, published 1997 in the InternationalConference On Multimedia Computing Systems, discloses a system oforganization of multimedia documents based on creating and categorizingmetadata derived from the content of the document. Known documents areplaced into folders as a method of access. These folders can then belinked together to associate their content. A delete operation isdefined on a folder where the operation will also delete the documentfrom subordinate folders.

While the above systems provide some limited ability to processmultimedia documents, it is desirable to have automated systems managingthe removal of such documents from the environment where appropriatewith no or limited user intervention required to locate and deletephysical and electronic content related to the document. Additionally,the above systems have a narrow view of the scope of what constitutes adocument to be processed and are essentially closed systems in that theyare solely responsible for managing the documents. Moreover, it isdesirable to be able to manage the removal and shredding of multimodaldocuments which, as explained below, may involve all forms of virtualmedia and physical media.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a multimodaldocument management system that allows the management of multimodaldocuments across different domains of control.

It is a further object of the present invention to enable the managementof shredding of multimodal documents including virtual media andphysical media relating to a multimodal document.

It is yet another objective of the present invention to provide a systemto interact with different domains of control to automate, based ontracking information available to the system, the shredding ofmultimodal documents by the deletion of virtual media and destruction ofphysical media relating to a multimodal document.

A multimodal document management system embodying the present inventionincludes a context manager for managing multimodal documents. A virtualmedia content management system is coupled to the context manager and aphysical media content management system is coupled to the contextmanager. The context manager is operable to issue multimodal documentmanagement instructions to the virtual media content management systemand issuing instructions to the physical media content managementsystem.

In accordance with a feature of the present invention, the contextmanager instructions relate to shredding of multimodal documents bydeleting virtual media and destroying physical media.

A method for managing multimodal documents embodying the presentinvention includes the steps of receiving a request for the shred ofelements in an information and content data file list of virtual mediaand physical media elements relating to a multimodal document. Adetermination is made if the request for shred is authorized and, wherethe request for shred is authorized creating a record of the shredrequest. Shred requested elements in the element list are grouped basedon which are managed by the same content management system. A message issent to each content management system requesting the shredding of eachelement of each grouping of elements managed by such same contentmanagement system.

DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate presently preferred embodiments ofthe invention, and together with the general description given above andthe detailed description of the preferred embodiments given below, serveto explain the principles of the invention. As shown throughout thedrawings, like reference numerals designate like or corresponding parts.

FIG. 1 is a block diagram of a system for deleting multimodal documentmanagement embodying the present invention which manages the process ofshredding (deletion and/or destruction, as the case may be) of variousforms of related information which can be embedded in virtual mediaand/or physical media;

FIG. 2 is a diagrammatic illustration of an example of the system shownin FIG. 1 involving multiple information management domains;

FIG. 3 is a flow chart of the operation of the system shown in FIG. 1involving the initiation of the process for shredding a multimodaldocument where the specific document is a physical document (physicalmedia) enhanced with on-line annotation (virtual media);

FIG. 4 is a view of a computer screen showing a cover sheet for thephysical document referenced in FIG. 3 having a shredding enablementarea, here shown as a personal shredding enablement area;

FIG. 5 is a flow chart of the operation of the system shown in FIG. 1involving the shredding process of the specific physical documentreferenced in FIG. 3;

FIG. 6 is a flow chart of the operation of the system shown in FIG. 1for initiating the process for shredding (deletion and destruction)involving the complete set of elements in an information and contentdata file for another multimodal document, here an exhaustive shreddingincluding all related virtual media and physical media and all usecontext related to the multimodal document;

FIG. 7 is a view of a computer screen having three open windows helpfulin an understanding of the operation of the system shown in FIG. 1 wherewindow A illustrates information and content data which list all of thevirtual media (virtual document and parts) and all of the physical media(physical document and artifacts) for the multimodal document, window Billustrates a pdf document listed as the second electronic document inwindow A) and window C illustrates of one of the virtual document partslisted in Window A;

FIG. 8 is a complete view of window A shown in FIG. 7 unobstructed byother open computer screen windows where the complete information andcontent data is shown with the full listing of the all of the virtualmedia (virtual document and parts) and physical media (physical documentand artifacts) for the multimodal document;

FIG. 9 is a flow chart of the operation of the system shown in FIG. 1 inthe comprehensive shredding process of a multimodal document with theexhaustive deletion and destruction of the complete set of informationand content data relating to the multimodal document where all virtualmedia (virtual documents and parts) and all physical media (physicaldocuments and artifacts) are to be deleted and/or destroyed and allrelated context removed;

FIG. 10 is a more detailed flow chart of the operation of shreddingprocess shown in FIG. 9, providing further details of the operation ofthe system where the context manager organizes and initiates theexhaustive deletion and destruction of the multimodal document;

FIG. 11 is a more detailed flow chart of the operation of shreddingprocess shown in FIG. 9, providing further details of the operation ofthe system where the context manager awaits responses from variousmultimodal document content management servers and users; and,

FIG. 12 is a more detailed flow chart of the operation of shreddingprocess shown in FIG. 9, providing further details of the operation ofthe system where the context manager initiates action when the timeallotted to receive a response from a content management server and/oruser has passed.

DETAILED DESCRIPTION OF THE PRESENT INVENTION

Various terms used herein have the meaning and usage set out below.

Multimodal documents include all forms and types of information. Whilemultimodal documents are often physical documents and related virtualinformation, it also includes both virtual media and physical media.Virtual media, for example, include electronic versions of documents,e-mails, and metadata stored in memory of various types, electronicversions of video information stored in various forms, electronicversions of audio information stored in various forms and softwarestored in various electronic forms. Physical media, for example, includeprinted documents, media containing video information (DVDs, tapes,etc.), media containing audio information (CDs, tapes, etc.), physicalmodels, photographs, and software disks. Multimodal documents contentmay be rendered, for example, in complete documents, artifacts and partsthereof that can exist in paper or electronic form. Virtual media andphysical media are often related in that, for example, a printeddocument with hand written annotations (physical media) thereon may beelectronically stored (virtual media). These media, the physical mediaand the virtual media, are related in that they involve the same orsimilar information or information that has been modified. For example,an electronic copy of an engineering drawing (virtual media) and thephysical embodiment depicted in the engineering drawing (physical media)are both related in the information they convey.

Information domains are the various environments in which the virtualmedia and the physical media exist and are managed.

Domains of control are the various systems and/or subsystems, such as acontent management server (server and associated software), responsiblefor control of a certain set of information such as video files, audiofiles, document metadata, or physical and/or electronic renderings of adocument.

Information and content data is the complete set of data includingcontent, use and domains of a multimodal document including virtualmedia and physical media; essentially a bucket of data pertaining to themultimodal document. The data, typically a list of elements, can bestored in a file or a database or otherwise. Herein, the phraseinformation and content data file (ICDF) is intended to encompass all ofthe foregoing.

Shredding includes the deletion of virtual media and the destruction ofphysical media, as the case may be.

The following is an overview of the operation of the system shown inFIGS. 1-12. The system facilitates and enhances the ability to moretotally shred and also to automate the shredding of multimodaldocuments. Once a request for shredding is made, the system will verifythe requesting entity is allowed to shred the given multimodal documentor portions of the multimodal document. If authorized, the system willshred all, or as much as is authorized or requested, of mediaconstituting the multimodal document including media created in theproduction of the document. Certain pieces of such a multimodal documentmay be stored on different systems or in different locations. For eachpiece of the multimodal document, a request or instruction for deletionof that specific piece is issued. The request for deletion can begrouped. The request is communicated to and processed by the systemresponsible for that particular piece of the multimodal document, andthe success or failure of the deletion is forwarded to the user orentity requesting shredding. The system can be indifferent to the typesof media to be shredded or treat different types of media differently.The system is operable across various information domains includingvirtual media domains and physical media domains. Thus, for example,video content is treated the same as audio content, which is alsotreated the same as a paper document or an annotated paper document.

Reference is now made to FIG. 1. A user accesses, through an inputsystem 2, a context manager 4 which manages the tracking and shreddingof both virtual and physical media. The context manager interacts withone or more system(s) 6 for tracking virtual media and one or moresystem(s) 8 for tracking physical media of multimodal documents. Thesystems 6 and 8 are content management systems and, in part, operatebased on instructions from the context manager 4 with respect tomultimodal documents. Each of these systems can be separate fullfunctioning system which provides a variety of independent operationalfunctions such as printing documents, word processing, video creationand processing, audio creation and processing. These systems can operatein separate domains and be controlled independently by various meanssuch as personal computers, servers, multifunctional devices, smartpaper shredders, etc.

The virtual media content management system 6 is coupled to a virtualmedia shredding subsystem 10 which shreds, by deletion, the virtualmedia. The physical media content management system 8 is connected to aphysical shredding subsystem 12, which shreds by destroying the physicalmedia. The context manager 4 interactively operates with the variousvirtual media content management system(s) and the physical mediacontent management system(s) to both identify and compile the content ofinformation files for multimodal documents, to issue shreddinginstructions and to track the completion of the various shreddingoperations. Although shown as separate shredding systems, subsystems 10and 12, each of these subsystems can be part of the content managementsystem(s) for tracking media to which it is connected.

Reference is now made to FIG. 2. The context manager 4 includes astorage means 14 where the information and content data for multimodaldocuments (entity and context models) and other related information arestored. The context manager 4 is coupled via any of a large number ofcommunications systems, here shown as messaging space 16, to variousdevices and servers. The context manager 4 is connected so that it canspan physical locations and various organizations and structures tomanage the totality of the multimodal document content and storereferences to that content in an information and content data file. Themultiple domains of use for the multimodal document, as shown in FIG. 2,is such to enable access by the context manager 4 to a plurality ofcommon, potentially associated (such as video and audio), messagingsystems and, for example, by shared names for objects that are involvedin the various environments. This can include people, devices anddocuments, both virtual and physical. The domains to which the contextmanager 4 is connected each incorporate software systems, instrumenteddevices, spaces and/or procedures that help to create and trackmultimodal document use.

The context manager 4 is coupled via the messaging space tomultifunctional devices 18 and 20. These may be devices that have beenenhanced with one or more document identification technologies, such asradio frequency identification device (RFID) readers, bar code/ANOTODocld capture, document image capture and hashing. These types oftechnologies, and others, are employed so that they can identify apre-existing document that is being printed, scanned or copied.Additionally, these multi-functional devices can create document clonesthat have been uniquely named, using one of more of these technologies.A printer can create a physical clone of an electronic document while ascanner will do the reverse by taking a physical document and creatingan electronic document.

Shredders 22 and 24 may be normal shredders that have been enhanced withone or more document identification technologies, such as those notedabove. Such shredders can operate to inform the context manager 4, andother subsystems as appropriate, via the messaging space 16, when adocument has been shredded, that is, physically destroyed, and, shouldthe shredder have a memory, the virtual shredding, that is, the deletionof any relevant information in the shredder related to the multimodaldocument. An example of one type of shredder with enhanced documentidentification capability is disclosed in U.S. patent application Ser.No. 11/156,127, filed for B. D. Singer et al. on Jun. 17, 2006, entitledSYSTEM AND METHOD FOR CONTROLLING THE STORAGE AND DESTRUCTION OFDOCUMENTS and assigned to Pitney Bowes Inc. (Pitney Bowes Docket No.F-933). Active workspaces 26 and 28 may be desks, shelves or otherphysical places where documents are used that may be enhanced with oneor more document identification technologies. These active workspaces 26and 28 can have additional on-line, active spaces, such as a user'selectronic document mailbox, that act similarly to the physical spaces.

There are two primary types of software systems (audio server and inkserver) shown in FIG. 2. These systems provide media services and asystem that senses and represents reports events associated with thecontent, that is, document use within the environment. The audio server30 manages all of the captured audio media data stream and stores neededinformation either in its own system storage 32 or on a locally accessedstorage. The audio server 30 need not be physically co-located withother subsystems or other domains. Access to the audio server 30 bydifferent users may be implemented through various types of audio inputcapability, shown at 34, 36 and 38. The ink server 40, like the audioserver 30, manages all of the ink media data stream and stores neededinformation either in its own system storage 42 or through locallyaccessed storage. The ink server 40 need not be physically co-locatedwith other subsystems or domains being managed by the context manager 4via the messaging space 16. The ink server 40 can employ digital pens,such as ANOTO pens 44, 46 and 48. The ANOTO pens 44, 46 and 48 may be onusers' personal computing space and interconnected to the ink server 40.

There can, of course, be other domains, including other media serverssuch as those relating to video domains. The architecture of the systemshown in FIGS. 1 and 2 can be modified, provided it has thefunctionality of enabling the context manager 4 to communicate directlyor indirectly with the various information domains it is expected tomanage in the tracking and shredding of multimodal documents. Eachinformation domain has a system(s) which provide a domain of control tomanage the set of content for which it is responsible. For thearchitecture shown, various additional domains and subsystems may beadded. Moreover, the context manager can operate within a secureenvironment if that is desired, such as a secure building, as denoted bythe line 50 encircling the context manager 4 and all of the variousdomains of control.

As can be seen above, the context manager 4 provides multimodal documentcompilation. It communicates via the messaging space 16 for messages anddata emanating from all of the various information domains itinteroperates with and derives the use of each managed multimodaldocument in each managed domain. For example, when a managed multimodaldocument is physically printed, the context manager 4 is informed viathe messaging space of this activity. The context manager 4 uses thevarious messages communicated to it via the messaging space to recordmultimodal document use information. This can include the creation ofdocument clones via the multifunctional devices. The context manager 4can obtain information relating to the physical location of documents,such as via the active workspaces, and the attachment of media clips onvirtual or physical documents via the various media servers, such as theaudio server 30 and the ink server 40. There may also be other on-lineinternet or intranet applications, not shown, that communicate with thecontext manager 4, such as word processing document used on a personalcomputer.

For the architecture shown in FIG. 2, the context manager 4 is thesingle point of control and relies on the various subsystems in thevarious domains to shred the multimodal documents. As shown in FIG. 2,each of the various servers manages the use of the various activitiesrelated to that server. For example, the audio server 30 manages audioclip information but communicates that management functionality via themessaging space 16 to the context manager 4. Thus, the context manager 4remains the single point of control for tracking, issuing shred commandsand monitoring shredding activity. Other architecture can also beimplemented where the single point of control is modified to provideadditional or separate points of control for separate types ofsubsystems or separate categories of multimodal document virtual mediaand/or physical media.

The system mitigates the need for human intervention when shredding amultimodal document in the system. The multimodal document is shreddedto the best ability of the system based on available data concerning themultimodal document and the use of the multimodal document. Theshredding of virtual media such as computer files existing on a mediaserver in one location and physical media such as printed documents inanother location can be brought about by a request from a contextmanager such as a server in yet another location. The context server mayalso possess greater knowledge of the context than individual systemsand users of the multimodal document because of the span of domainsabout which the context server stores or can access data. The systemuses this knowledge from the span of domain to assist users incompletely shredding a multimodal document or portion of such documentas desired and authorized.

The system is organized for the context manager to communicate withvarious domains of control related to the multimodal document so that itis able to initiate and, depending on how the system is implemented,monitor and/or confirm, the shredding of a multimodal document includingrenditions and annotations associated with the document. Upon receivinga request or instruction for shredding, a domain of control, such as aserver, will attempt to shred the portions of the multimodal documentwhich it processes or controls. The domain of control can communicateback to the system the completion of the shred and/or the status of anattempted shred. After receiving responses to various requests, orexpiration of a timeout period, the system informs affected users thestatus of the shredding of the multimodal document. If the system hasbeen unable to shred the content to the extent requested, thesenotifications of shred status from the various domains may function as arequest for user intervention. If successful, the notifications insteadsignal the completion of the shredding process to the user.

Thus, the system manages documents that exist in both the physical andelectronic environment. For any given document managed by the contextmanager, the shredding process ensures the removal of the document'smetadata and all its associated content (annotations and renderings). Arendering is analogous to a copy of the document. Just as copies of adocument can exist in a paper form or as an electronic file, renderingshave an equal ambivalence to medium. The system proactively managespaper documents, for example, in several ways. First, if a document isdeleted, the system will alert users to the deletion event and requestthat they destroy the paper rendering. Second, the system will monitoruser interactions with a paper rendering for an action that representsdeletion of that document. On the occurrence of the triggering process,the system will begin a deletion of all content and metadata related tothe current document.

Reference is now made to FIG. 3. An ANOTO-ized document is created at 52with a document cover sheet. The document is registered at 54 with thecontext management system. The document is annotated with ink marks,audio and other media stream segments at 56. This is in accordance withthe operation of the various subsystems as described in FIG. 2. When ashredding operation is desired, a check mark or indication is placednext or in the shredding enablement box or area. Here, the box ischecked for a personal shredding at 58. The personal shreddingenablement may be, for example, as shown in FIG. 4. Other more extensiveshredding enablement boxes or areas for various users can be provideddepending on the design of the system.

Reference is now made to FIG. 4. The document cover sheet referenced inblock 52 of FIG. 3 may be as is shown in FIG. 4. The computer view ofthe cover sheet 60 shows a shredding enablement area shown generally at62. The shredding enablement area 62 is a personal shredding enablementarea activated by the activation of personal shredding box 64. When thebox 64 is checked, all electronic references to the user's copy of thedocument will be deleted and the user will receive a confirmation e-mailonce the shredding process is completed. The personal shredding,depending upon how the system is implemented, can also include shreddingof all virtual media and physical media that involve the user's useand/or relationship to the multimodal document. Confirmation of theshredding, including physical destruction of any physical media as wellas the confirmation of the deletion of all virtual media can also beprovided. As previously noted, other more extensive and comprehensiveshredding enablement options can be provided, depending on the system.This can be implemented with verification of the authorization of theinitiator for each of the different aspects or parts of the document tobe shredded. For example, a manager may be allowed and authorized toinitiate the shredding of document parts used within the manager's teambut not allowed to shred documents outside of the team. An explicitfield, for example either in an information and content data file ordatabase (ICDF) at the context manager 4 or in a content managementsystem, can be provided which indicates who is authorized to shred andwhat they are authorized to shred. The field or another field can alsobe provided to indicate the ownership or custodian of what is to beshredded and where desired provide this information to other systems.

Reference is now made to FIG. 5, showing the operation of the systeminvolved in the shredding process of the specific physical documentreferenced in FIG. 3. At 66, a document is allowed to be shredded by arequester, as identified, for example, via the owner (user) of aspecific ANOTO pen. A shred request is sent to the ANOTO server at 68,including the identification that the shred request was registered tothe specific document to be shredded. At block 72, a shred request issent by the context manager 4 to each of the media servers for whichthere are media stream segments included in the metadata associated withthe document. At block 74, a countdown timer is started and at block 76,responses are awaited from each of the servers to which a shred requestwas sent by the context manager 4. At block 78, for all failed shreds,that is, a shred that has not been implemented or accomplished, then therequester is notified that there was a problem with that part of theshred operation. At block 80, for all successful shred operations, theassociated metadata from the context memory 4 is removed. At block 82, amessage is sent to the requester with the state of the shred of themultimodal document requested to be shred. The requester then mayphysically shred the personal copy of the requester's multimodaldocument.

Reference is now made to FIG. 6, where the process for initiating theshredding involving the complete set of content, as represented, theentire information and content data file, for a multimodal document isinitiated. At block 86, a information and content data file or database(ICDF) is created. At block 88, as content is created, it is registeredand reference is added to the information and content data file ordatabase and appropriately stored by the context manager 4. The contextmanager 4 tracks the use of each piece of content at block 90. At block92, an exhaustive shred is initiated, such as by right clicking theinformation and content data file for the multimodal document at thecontext manager 4 and selecting Exhaustive Shred shown in FIG. 8.

The tracking the use of each piece of content at 90 may include locationtracking of physical items and use tracking, like writing on items,mailing documents (e.g., e-mail sniffers and enhanced postal meters,such as meters with scanners), etc. All tracking information of bothphysical media and virtual media is sent to and managed by the contextmanager 4. The tracking methods can employ any of the methods describedabove and other well known methods useful in tracking and sensing andimage recognition, including by use of digital pens, radio frequencyidentification (RFID), bar codes and numeric fields, electronic documentidentification (ID) sensing, image recognition via cameras, and thelike.

Reference is now made to FIG. 7. A computer screen 13 exhibits threeopen windows, window A, window B, and window C. These windows areexamples of the type of information that is gathered, managed by anddisplayed using the context manager 4. In window A, an information andcontent data file illustrates and lists all of the virtual media(virtual documents and parts) and all of the physical media (physicaldocuments and artifacts) for a multimodal document. The information andcontent data file in window A includes three electronic documents 94, 96and 98. These three electronic documents 94, 96 and 98 are pdf-generatedfrom the original ANOTO document and include all existing annotations atthe time of generation. Two physical ANOTO-ized documents 100 and 102are also listed. These documents are Deb's copy and Jill's copy. Alsolisted are five virtual copy parts of multimodal document. These copyparts are shown as 104, 106, 108, 110 and 112. It should be recognizedthat depending upon the nature and use of the multimodal document, othertypes of listings from various subsystems may exist, such as audio clipsand video clips, etc.

Window B illustrates a pdf document listed as the second electronicversion 94 in Window A. This is document is designated as document 102 ain Window B. Window C illustrates one of the virtual document partslisted in Window A, here designated as 104 a. Various types and formatsof displays and organization of the various information related to thecreation, tracking and shredding of virtual media and physical media canbe employed, depending on the nature and design of the system.

Reference is now made to FIG. 8, which provides a complete view ofwindow A shown in FIG. 7 unobstructed by other open screen windows.Here, the complete information and content data file for the multimodaldocument is shown with a full and complete listing of all of the virtualmedia (virtual document and parts) and all of the physical media(physical documents and artifacts) at 94 through 112. Additional typesof information that can be further included and listed in theinformation and content data file. A user could initiate an exhaustiveshred of the full information and content listed in the information andcontent data file by activating box 95.

Reference is now made to FIG. 9 which shows the operation of the systemin the comprehensive shredding process of a multimodal document at ahigh level and with the exhaustive deletion and destruction of thecomplete set of content. At block 114, if the content is allowed to beshredded by the requester, as identified via the ownership of theinformation and content data file. At block 116, a shred request is sentto all the servers and administrators that manage the content items ofthe document, the context manager and content management systems. Thiswould include all of the systems shown in FIGS. 1 and 2, if that is theparticular architecture that is implemented. At block 118, a countdowntimer is started and at block 120 the system awaits responses from eachserver and/or administrator to which a shred request was sent.

At block 122, the requester is notified that there was a problem withall failed shreds. For all successful shred operations at block 124, theassociated metadata is removed from the context manager memory, storagemeans 4. A message is sent at block 126 to the requester with the stateof the shred operation. At block 128, the requester must explicitlyfollow up with external parties to ensure that their shred was complete,to the extent that the external parties are not within the domains thatare managed by the context manager 2.

Reference is now made to FIG. 10, where the context manager 4orchestrates a comprehensive shredding operation. The context manager 4receives a request for the comprehensive (exhaustive) shredding of thevirtual media and physical media listed in the information and contentdata file at block 130. This request may have been initiated by the useras indicated in FIG. 8, or may be initiated via some other interfacethat may be intentionally activated, for example, one similar to thedocument cover sheet shown in FIG. 4. A determination is made at block132 if the requester is authorized to remove the content from theinformation and content data file. Where the requester is not soauthorized, the context manager at block 134 discards the request. Therequest by an unauthorized requester for shredding may be stored, ifdesired, in the context manager 4 for subsequent audit and/or otheractivities. Where the requester is authorized to remove the content fromthe information and content data file at block 132, the context manager4 creates a record of the request at block 133. Thereafter, all elementsof the information and content data file are listed at block 134 if thelist has not previously been compiled. This includes clones, artifacts,document parts, and the like. The list of the context createdinformation and content data file includes all virtual media and allphysical media if the list has not previously been compiled. At block136, a list of all current users is created of all of the virtual mediaand all of the physical media. As noted, the list of elements and users(and any other list) can be created and exist prior to the request forshred being received. The list build can be ongoing and continuous aselements are created or the build can be periodic, that is, hourly,daily, weekly, monthly, etc.

At block 138, the context manager 4 sub-groups various elements in thelist. This may be based on the content management systems employed, suchas the various media servers, the various element inventoryadministration, users of the multimodal documents, and the like. Atblock 140, a message is sent to each content management systemrequesting the deletion and destruction of each element; for example,e-mails sent to persons and the like. At block 142, a response timer isset for each content management system. The response timer and timeinvolved can be unique to the particular content management system andeven within subsystems of the particular content management system,depending on the need and applications for the system. At block 144 alist of all items that have been sent out of the system domain iscreated. At block 146, the requester is notified of all such itemsincluding recipients of the items, and at block 148, the requesterfollows up on those items using traditional methods, e.g., phone mail,e-mail, and other appropriate methods to ensure the deletion ordestruction of the virtual media and physical media. The process steps144, 146 and 148 are for those portions of multimodal document that maynot or are not under the management of the context manager 4 and needseparate, independent operations to ensure their deletion and/ordestruction. Thus, the context manager 4 manages not only systems withinthe direct or indirect control of the context manager system 4 but alsorelated multimodal document virtual media and physical media related tosuch other materials that need to be independently processed when thatinformation is available to the context manager 4.

Reference is now made to FIG. 11, where the context manager 4 isawaiting a response from the various subsystems for which a shredrequest was issued. The context manager at block 150 awaits the nextresponse message from the content management servers and/or users. Foreach element included at block 152, a determination is made as towhether the requested shred request was accomplished. If the shredrequest was accomplished, the elements response trigger is disabled atblock 154 and the element is removed from the information and contentdata file list. At block 155 related context is removed. A determinationis then made at block 156 if the information and content data file listis empty. Where the information and content data is not empty, thesystem loops back to block 150. Where, at block 152, the shred requestwas not accomplished, a determination is made at block 158 for thespecific element that was not shredded whether the maximum number ofretries have been attempted. The number of tries is a matter of designchoice and can vary for each type of element to be shredded. Where themaximum number of retries have been attempted, the element is marked asa failed shred attempt at block 160. Where the maximum number of retrieshave not been attempted, at block 162 a request is sent to the contextmanager 4 to initiate another attempt to shred the same element. Whereat block 156 a determination is made that the information and contentdata file is empty, at block 164 a message, such as an e-mail, is sentto the person who requested the comprehensive deletion/destructionrequest, the shred request, notifying the requester that the requestedshred was successful.

Reference is now made to FIG. 12, where the absence of an expectedresponse from each content management system which was issued a shredinstruction is tracked by the context manager 4. At block 166, anelement response trigger fires. This is when the time allotted toreceive a response that a shred request has been accomplished haspassed. A determination is then made at block 168 if the maximum numberof retries have been attempted. Where this is not the case, at block 170a request is again sent by the context manager 4 to the content serverfor the element to delete and/or destroy, shred, the element. When themaximum number of retries have been attempted, the element is marked asfailed at block 172. The element is thus marked as an element of themultimodal document that has failed to be shredded on a shred request. Adetermination is then made at block 174 whether the list contains onlyfailed items. Where the list does contain only failed items, a messageis sent at block 176 to the person who requested the comprehensiveshredding, deletion and/or destruction request, notifying that therequest was not successful, including the list of elements that failed.Where the list does not contain only failed items, the system loops backto block 166.

While the present invention has been disclosed and described withreference to a single embodiment thereof, it will be apparent, as notedabove that variations and modifications may be made therein. It is,thus, intended in the following claims to cover each variation andmodification that falls within the true spirit and scope of the presentinvention.

1. A multimodal document management system, comprising: a contextmanager for managing multimodal documents; a virtual media contentmanagement system, said virtual media content management system coupledto said context manager; a physical media content management system,said physical media content management system coupled to said contextmanager; and, said context manager operable to issue multimodal documentmanagement instructions to said virtual media content management systemand to said physical media content management system.
 2. A multimodaldocument management system as defined in claim 1, further comprising: asecond virtual media content management system, said virtual mediacontent management system coupled to said context manager; and, saidfirst and said second virtual media content management systems eachoperating in different domains of control.
 3. A multimodal documentmanagement system as defined in claim 1, further comprising: a secondphysical media content management system, said physical media contentmanagement system coupled to said context manager; and, said first andsaid second physical media content management systems each operating indifferent domains of control.
 4. A multimodal document management systemas defined in claim 1, further comprising: a second virtual mediacontent management system, said virtual media content management systemcoupled to said context manager; said first and said second virtualmedia content management systems each operating in different domains ofcontrol; a second physical media content management system, saidphysical media content management system coupled to said contextmanager; and, said first and said second physical media contentmanagement systems each operating in different domains of control.
 5. Asystem multimodal document management as defined in claim 1, where saidvirtual media content management system is interactively coupled to saidcontext manager and said physical media content management system isinteractively coupled to said context manager such that said virtualmedia content management system and said physical media contentmanagement system can interactively communicate with said contextmanager.
 6. A multimodal document management system as defined in claim5, further comprising: a virtual media shredding subsystem coupled tosaid virtual media content management system and a physical mediashredding subsystem coupled to said physical media content managementsystem.
 7. A multimodal document management system as defined in claim6, wherein said virtual media shredding subsystem is operable undercontrol of said virtual media content management system to deletevirtual media upon instructions from said context manager to saidvirtual media content management system and wherein said physical mediashredding subsystem is operable under control of said physical mediacontent management system to destroy physical media upon instructionsfrom said context manager to said physical media content managementsystem.
 8. A multimodal document management system as defined in claim7, wherein said context manager receives communications from saidvirtual media tracking system concerning the status of virtual mediarelated to multimodal documents and receives communications from saidphysical media tracking system concerning the status of physical mediarelated to said multimodal documents.
 9. A multimodal documentmanagement system as defined in claim 8, wherein said context managercreates information and content data files concerning multimodaldocuments.
 10. A multimodal document management system as defined inclaim 9, wherein said context manager information and content data filesfor a multimodal document includes a list of virtual media contentrelating to said multimodal document and of physical media contentrelating to said multimodal document.
 11. A multimodal documentmanagement system as defined in claim 10, wherein said information andcontent data file is employed in identifying elements of said multimodaldocument in issuing shredding instructions.
 12. The multimodal documentmanagement system as defined in claim 11, wherein said shreddinginstructions issued by said context manager to virtual media contentmanagement system and to said physical media content management systeminclude personal shredding instructions involving the shredding of allmultimodal document element content in said information and content datafile relating to a particular user.
 13. The multimodal documentmanagement system as defined in claim 11, wherein said shreddinginstructions involving exhaustive shredding of all multimodal documentelement content in said information and content data file.
 14. Amultimodal document management system as defined in claim 12, whereinsaid context manager subsequent to the shredding of all said personalvirtual media relating to a multimodal document and all said personalphysical media relating to a multimodal document deletes all personalvirtual media data relating to said multimodal document stored in memorymeans for said context manager.
 15. A multimodal document managementsystem as defined in claim 13, wherein said context manager subsequentto the exhaustive shredding of all said virtual media relating to amultimodal document and all said physical media relating to a multimodaldocument deletes all virtual media data relating to said multimodaldocument stored in memory means for said context manager.
 16. Amultimodal document management system as defined in claim 11, whereinsaid multimodal document includes physical media including physicaldocuments and virtual media including virtual documents relating to saidphysical documents.
 17. A multimodal document management system asdefined in claim 16, wherein said multimodal document virtual mediafurther includes audio clip media relating to said physical documents.18. A method for managing multimodal documents, comprising the steps of:receiving a request for a shred of elements in an information andcontent data file containing a list of virtual media and physical mediaelements relating to a multimodal document; determining if the requestfor shred is authorized and, where said request for shred is authorized,creating a record of said shred request; grouping said shred requestedelements in said element list based on each element in said which aremanaged by the same content management system; and, sending a message toeach content management system requesting the shredding of each elementin each grouping of elements managed by said same content managementsystem.
 19. A method for managing multimodal documents as defined inclaim 18, comprising the further step of setting a response timer forreceiving a response to said shredding request message from each contentmanagement system to which a shredding request is sent.
 20. A method formanaging multimodal documents as defined in claim 18, comprising thefurther step of: awaiting a response from said content managementsystems; determining if the shred request was accomplished from eachcontent management system response; disabling an element responsetrigger and removing each element from said list where a response isreceived from said content management system that said shred request wasaccomplished, where said element response trigger is triggered by afailure to receive said response; and, if said information and contentdata file list elements have been successfully shredded, sending amessage to said requester that said shred request was successful.
 21. Amethod for managing multimodal documents as defined in claim 20,comprising the further step of repeating said steps beginning with saidawaiting a response from said content management systems if saidinformation and content data file have not been successfully shredded.22. A method for managing multimodal documents as defined in claim 21,comprising the further steps of, determining if a predetermined maximumnumber of retries have been attempted for shred requested elements wheresaid element response trigger has fired when a response period haspassed for a content management server to respond; and, where saidmaximum number of retries attempts have been implemented for shredrequested elements, marking said element as having failed to beshredded; and, notifying said requester that said element has failed tobe shredded.
 23. A method for managing multimodal documents as definedin claim 18 wherein said request for a shred of elements in aninformation and content data file containing a list of virtual media andphysical media elements relating to a multimodal document is a requestfor a comprehensive shred of all said elements of virtual media andphysical in said information and content data file list.
 24. Amultimodal document management system, comprising: a context manager formanaging multimodal documents; a first and a second virtual mediacontent management system, said first and said second virtual mediacontent management system coupled to said context manager, said firstand said second virtual media content management systems each operatingin different domains of control; a first and a second physical mediacontent management system, said first and said second physical mediacontent management system coupled to said context manager, said firstand said second physical media content management systems each operatingin different domains of control; said context manager operable to issuemultimodal document management shredding instructions to said first andsaid second virtual media content management system and to issueshredding instructions to said first and said second physical mediacontent management system; and, said first and said second virtual mediacontent management system, in response to shredding instruction receivedfrom said context manager, operable to initiate the process of shreddingof virtual media; and, said first and said second physical mediatracking system, in response to shredding instruction received from saidcontext manager, operable to initiate the process of shredding ofphysical media.
 25. A multimodal document management system as definedin claim 24, wherein said context manager receives communications fromsaid first and said second virtual media tracking system concerning thestatus of virtual media shredding related to multimodal documents andreceives communications from said first and said second physical mediatracking system concerning the status of physical media shreddingrelated to said multimodal documents.